fix(native-eval): bootstrap only the assigned harness - #71
Merged
Conversation
|
🦞👀 Pull request received. I will update this pull request when review starts. |
|
ClawSweeper status: review started. I am starting a fresh review of this pull request: fix(native-eval): bootstrap only the assigned harness This is item 1/1 in the current shard. Shard 0/1. This placeholder means the worker is alive and reading the current context. I will edit this same comment with the actual review when the claws are done clicking. Crustacean status: shell secured, claws on keyboard, evidence pebbles being sorted. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Fleet knows the harness assigned to each lease, but hydration discarded that selection. The bootstrap script consequently installed OpenClaw, Codex, Claude Code, and Hermes on every machine, even for a single-harness campaign.
Change
Pass the run's harness into bootstrap and require exactly one supported selection. Keep Docker, Node, uv, and LiteLLM shared; install and probe only the selected harness. Move uv's managed-Python/cache configuration out of Hermes setup so the shared LiteLLM environment remains self-contained when Hermes is not installed.
The manifest retains the selected harness's existing version key and adds
harness; uninstalled harness keys are absent. Manual bootstrap now requires a harness argument. The research runbook and changelog document this behavior.Fixes #65.
Proof
bootstrap_beast.sh openclaw: exit 0 in 171.542 seconds. Installed OpenClaw2026.7.1-2 (0790d9f), Nodev22.23.1, and LiteLLM1.93.0. Ran the real OpenClaw CLI and LiteLLM Python environment, asserted Codex/Claude Code/Hermes directories absent, and verified the manifest contains only OpenClaw plus shared versions.The AWS smoke followed Testbox teardown/concurrency trouble and a Testbox console-script PATH failure after its suite and wheel build had passed. This is installer proof, not a model-evaluation run or a measured bootstrap speedup. Actual installation was exercised for OpenClaw; the other three selections have executable shell regression coverage.